Avoid accidental FP64 promotion in UMAP, t-SNE and HDBSCAN float kernels - #8538

Summary

A handful of float kernels in UMAP, t-SNE and HDBSCAN contain unsuffixed double literals (0.5, 1e-8, pow(x, 2.0 * b)) or double locals. C++ promotes the surrounding float expression to double, so nvcc emits FP64 instructions in kernels that are otherwise entirely single-precision.

This is close to free on datacenter parts, but consumer GPUs have heavily reduced FP64 throughput — 1/64 of FP32 on GeForce Blackwell — so the promoted arithmetic is disproportionately expensive there. This PR types those literals to value_t/T so the float instantiations stay in float.

Please read the benchmark section before treating this as a performance PR — the end-to-end gains are small, and zero for UMAP.

How the sites were found

Source grep alone is far too noisy (most double literals are compile-time conversions that nvcc folds away — e.g. smooth_knn_dist_kernel and optimize_batch_kernel look suspicious in source but emit no FP64 at all, and are deliberately left alone here).

Instead each candidate was compiled to sm_120 SASS, and FP64-class opcodes (DADD/DMUL/DFMA/DSETP/MUFU.RCP64H plus F2F.F64.F32/I2F.F64 conversions) were counted per kernel, keeping only kernels whose demangled signature contains no double. -lineinfo plus nvdisasm -g then attributed each instruction to a source line.

Across umap.cu, tsne.cu and the HDBSCAN TUs, FP64-class instructions in float-only kernels drop from 2173 to 12. The remaining 12 are in upstream raft::random Box-Muller code, outside this repo.

Kernels with double in their signature were checked separately: zero of them changed their FP64 count, so the double instantiations are unaffected.

Benchmarks (RTX 5090, CUDA 13.2, sm_120)

Per kernel the improvements are large:

Those numbers are misleading as a headline, though. Whole-run profiling shows the kernels this PR touches account for 3.89 ms of 2193.6 ms of GPU kernel time (0.18%) across a combined UMAP + t-SNE + HDBSCAN session. End to end, on 200k×64 synthetic blobs:

So realistically:

• UMAP: no measurable gain. The fixed kernels run once per fit and total 0.14 ms against a 777 ms fit; the time is in kNN construction and optimize_batch_kernel, which had no FP64 to begin with.
• t-SNE: 0–4%, depending on whether the run hits min_grad_norm early. IntegrationKernel and the gradient-norm transform run every iteration, so the saving scales with n_iter.
• HDBSCAN: no gain on the fit; up to ~6% on soft clustering, and only with many clusters, since that kernel is O(n × n_clusters). It is capped there because the pairwise-distance GEMM and the min-reduction dominate that call.

I would suggest taking this on correctness/hygiene grounds — unintended type promotion in a single-precision kernel is a bug even where it is cheap — rather than as a performance improvement. It is a small diff and it makes the intent of the arithmetic explicit.

Numerics

Output buffers were fingerprinted (sum, sum-of-squares, max, NaN count) in both builds:

• t-SNE FunctionalSqrt: bit-identical
• HDBSCAN membership vectors and probabilities: bit-identical
• UMAP membership strength: 9e-9 relative on the sum over 7.5M values
• UMAP Optimize::f: identical to 10 significant digits
• no NaNs introduced anywhere

One caveat worth flagging: full-length t-SNE embeddings are no longer bit-reproducible against the previous build. Total spread agrees to 0.04% (sumsq 1.70359e8 vs 1.70430e8) and the embedding is equally good, but the changed rounding diverges over 1000 iterations. Tests that score trustworthiness are fine; anything pinning exact t-SNE coordinates would need regenerating.

One change is a deliberate reduction in working precision rather than a no-op: compute_membership_strength_kernel previously accumulated in double regardless of value_t, and now follows value_t like the rest of the fuzzy simplicial set construction. That is the change most worth a reviewer's eye.

Testing

I have not been able to run the C++ gtests (cpp/tests/sg/{umap_parametrizable,tsne,hdbscan}_test.cu) locally — my build is configured with a restricted CUML_ALGORITHMS set. Validation so far is the SASS analysis and the output fingerprinting above, so CI coverage on these three algorithms would be worth watching.

🤖 Generated with Claude Code